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(54) Image processing 

(57i The more modern imaging 
systems employ solid state detector 
area arrays which "stare" at the viewed 
scene (and are thus known as "staring 
arrays") Unfortunately, these arrays are 
not very uniform, and the sensitivity 
and the output absolute value of the 
detector elements can vary significantly 
over the array. A normalisation process 
can be employed, but if such a process 
is based on the assumption that each 
element's output is a linear function of 
its input there may be problems. The 
present invention suggests a solution to 
this difficulty; it proposes dividing the 
operating range of the elementsririto a 
plurality of adjacent regions, calculating 
the gain and offset correction factors for 
each region separately (upon the basis 
that the input/output relationship in that 
region is linear), and then in operation 
correcting the element's output in 
accordance to which region it is in. 
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SPECIFICATION 
Imag processing 

5 This invention concerns image processing, and relates in particular to the processing of image data derived 

with any real consistency and u "^2^^,XXnd*. output absolute value (dependent 
20 the sensitivity (the gain in output for a spec.f.ed gam in input) ana tn o P "offset") of 

upon both the gain and the output at ™J£Z^^™' £Sl t£e « Son is usual.y marked.y 
the detector elements can vary very ^ ^ 

;x h r w ruS 

25 provide - ultimately - a meaningful picture. imnrnvpments in the technology enabling detector arrays - 

The problem may in the long term £^^^^2^ thesame gain and offset. For 

especially IR arrays-to be constructed so that all the element m u « * N 832 94 7 o 

the'present, however, the inventior £^ detector element is 

(I/6768/ELL) puts forward quite a different solution, irv which ™ e ° u P jfically> the re are found for 

30 norma.isedlcorrectedtotheva.uethatana^ 

each detector element the values defining its gam and « he ™ ae , } and 

two factors that will normalise these values (convert ^^™£%£X« each element, before that 

thereafter the relevant two normalisation 1a «°*™ l "™^?£* so that the output is modified to the value 

^method oftheinventionofth^ 
40 under the same input conditions (and "average means a J^JJJ^,. array P that actually WO rk). 

45 change in output, while the offset u i the ar«thmet.cal1 actor «™S^™ ° a detector eleme nt's 
output may be described by the equation 



10 



Y = AT - B 

50 



(whereYistheoutpu.Tistheinput-an^ 

constants). A plot of Y for a range of values of T gives. i sjwght ne ° P es A times tnat chang e in 

(when T = 0) at B. The constant A is thus the gam of the e ement < a cha ^ e ^ = C J . when the e|ement sti „ 
55 Y), while the constant B is the offset (the conven.ent point is with zero input 

9 ^av:ra t g P e t ga!n A and average offset B for a series of n elements are 

n 

■E * 
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[Zl 5 
5 n 

and the mean output val ue Y* is thus 

Y»=A.T+B" <U 10 

10 

For any particular element (the ith one) : 

Y, = Aj.T + B; (2) 

15 

15 T = Y; - Bj 

so AT 

Combining (1) and (2); 20 

20 Y* =~A. (Yi- B;) + B 

A, 

= A .Y, + (B -A.B) 25 

25 a, a; 

or = K1,.Y, + K2, (3) 

30 where K1j =~Aand K2j =TT— KI.Bj 30 

A, 

Y*= K1|.Y, + K2j 

simultaneous equations 

Yi1 = AfT, + Bi 50 
50 Y i2 = A,T 2 + B, 

55 *J5£SS£d on the assumption that eact <-^"^^^^^ M nM««» 
Unfortunately, however, this may not always be so - and ^"'*™3»l«9 0«i some, or 
from a linear one that the gain and offset correction factors £ ^ ™V be Satively S 7™ie and easily 

correcting the element's output in accordance to "hWi «8«on individual gain and offset 

66 J£5£22S£^^ " 



5MSDCCID <GB 2149605A_L» 



GB2 149 605 A 



dement of an array of electromagnetic radiation detector elements used in an imaging system to provide an 

5 io reach^egion ^individual correction va.ues for eah element are calculated upon the basis that the I 
relationship between the element's input and output is linear thereover; 

whereby in operation each element's output value may be corrected us.ng the correction factors 
appropriate both to that element and that region of the range into which ,ts output value fa "f; 
In mother aspect, the invention provides a method of processing the output of an ele ^magnetic 
10 radiation detector element array to improve the quality of the image generated therefrom, in which method 11 
ZZr to ir m.S to aenerate the image the output of each detector element is first normalised by applying 
thTreto I?, gain and ZZ correction fa ^appropriate both to that element and to that region of the range 

"e^ , 

Z^nlZSrtM^MWO* either a silicon CCD structure or an array of MOS switches and specific 
xamp.es of tt !ype of array a're the Integrated Photomatrix Ltd. IPL2D1 (a 64 x 64 photod.ode array 
sensitive to visible light) and the 32 x 32 M4680 IR CCD element available from ^^- e|ementg „ 2 
20 In accordance with the invention the expected operating output value range for the detecto re lements .s 
divided into a plurality of regions. While naturally the whole output value range 

seems little point in having regions within the range that are in pract.ee never going to be used. Thus, the 
rSonsT e within the expected operating value range -though, of course, what this range ■* depends upon 
he SJ^t£m«.v« P .nd the norma, operating conditions! If by chance V^^™?*? pKt s 
9R of the ranae either above the highest region or below the lowest region (i.e., beyond either end of the 
25 expected opeTat'g range) .hen it can be arranged that it is treated, and thus corrected, as though it fell 

rather SS? actual, and occur merely at some arbitrary but predetermined values recognised (as expla med 
30 hereinafter) 5 *e associated electronics. As also explained hereinafter, each region is bounded by a range . 

relions e interrelated, and to a degree dependent upon the expected actual shape of the curve definmg the 
have been read out from the right location in the chosen * OT * an * th °™™^ apparatus 
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particularly acceptable method involves using a sequence of a corresponding plurality less one of 
comparators in which are permanently stored details of the output values defining the points where adjacent 
regions meet, and the actual output value is compared in all of these (either simultaneously or sequentially) 
and their outputs combined to indicate at which level - and thus in which region - the detector element's 

5 output value falls. This type of level assessment is itself quite conventional, and needs no further discussion 
here (though an example is described hereinafter with reference to the accompanying drawings). 

Apart from the feature of having a plurality of regions in the element's output value range, each region 
having its own gain and offset correction factors, the invention is much the same as that of the 
aforementioned Application No. 8329470. 

10 Accordingly:- 

The method of the invention - and specifically the normalisation calculations and the storage of the 
constants involved - is conveniently performed under the control of a microprocessor - for example, a Zilog 

The image-forming apparatus will normally be a visual display unit (VDU) of the cathode ray tube 
15 (television) type, which is "written" to frame by frame and field by field to produce the desired visible image, 
and the modified detector element array output is most preferably fed to the VDU via one or other of two 
temporary stores capable of holding the output values for one whole field at a time, with "dead" element 
correction. . . 

The invention has so far been described in terms of a method. In another aspect the invention provides 
20 apparatus for processing the output of an electromagnetic radiation detector element array to improve the 
quality of the image generated therefrom, which apparatus comprises means to normalise the output of 
each detector element prior to that output's use to generate the desired image, this normalising means being 
means for applying to each detector's output a gain correction factor and an offset correction factor each 
relevant to that particular detector, wherein: the factor-applying means itself includes (1 ) a plurality of stores 
25 wherein are stored the gain and offset correction factors, for each detector element, for a like plurality of 
regions into which the detector element's expected operating output value range is notionally divided, and 
(2) assessment means whereby each element's output value may be assessed and the appropriate 
correction factors for that value of the element identified. In practice, of course, the factor-applying means 
will also include a multiplier and adder (to which are applied the output value to be corrected and, 
30 respectively, the appropriate gain correction factor and the appropriate offset correction factor), an address 
generator (to identify the next element whose output is to be modified, and to identify the relevant set of 
correction factors) and a system driver (a clock). In its more preferred forms the apparatus also includes two 
video stores (for holding two sequential sets of modified detector arrayoutputs) and means to switch the 
input to those stores, and the output therefrom to an image generator, from one to the other, and very 
35 preferably there is further incorporated means to provide for any faulty element an average output value 
based upon the output values for its neighbouring elements. 

The invention extends, of course, to any imaging system, especially a thermal system, making use of the 
image-improving method or apparatus as described and claimed herein. 
An embodiment of the invention is now described, though by way of illustration only, with reference to the 
40 accompanying drawings in which:- 

Figure 1 is a schematic view of a thermal imaging system employing the method and apparatus ot tne 

mV ffgures2A and B are representational input/output plots showing various typical curves for detector 
element output values, and how these can be divided into regions; 
45 Figure 3 is a simplified block diagram showing the method and apparatus of the invention; 
Figure 4 is a more detailed block diagram of part of Figure 3; and 

Figure 5 is a more detailed version of all of Figure 3. _ 
In Figure 1 there is shown a thermal imaging camera (11 ) viewing a scene (12) and providing a visible 
image of that scene in a television-type Visual Display Unit (13) via a processing module (14). The camera 11 
50 employs an IR detector element array (not shown separately), and the processing module 1 4 normalises the 
output of that array in accordance with the invention to enable a meaningful image to be formed on the VDU 
13 The nature of the processing module 14 is shown in more detail in Figure 3. 

The input/output plots of Figures 2A and B show two possible curves (121A,B) describing the operating 
range of two different elements, and how each can be treated either as a segment of a single straight line or 
55 as a combination of segments of two or more straight lines. Taking Figure 2A first, the plot is the continuous 
heavy line 121A. On a very simplistic view this can be thought of as approximating to the straight line 
segment (the dashed line 1 22A) between two points (1 23A, 1 24A) near each end, but this is clearly a rather 
inaccurate approximation, and a much better one is to considerthe curve as two straight line segments (the 
dotted lines 125A, 126A) between the two points 123A and 124A and a third point (127 A) mid-way between 
60 them, these three points dividing the curve into two adjacent regions. , _u 

Obviously, the more the number of points between the two other points 123A and 1 24A the closer each 
line joining consecutive point pairs is to the curve itself. 

The "S" shaped curve 1 21 B in Figure 2B is a more complex curve than 121 A in Figure 2A, and needs more 
points, defining more regions, to ensure that the point-joining straight line segments give a good 
65 approximation. As in Figure 2A, a rough approximation is a straight line segment (dashed line 131 B) joining 
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• ♦eii- I 9R 1-wm near either end but a better approximation is obtained by dividing the curve into 

4 °°"«^aaaing.<ha2b^^ 

"^el ^".nav.n ™,a d«a,,ad b.oaUdiagran, C.ha Proaasaing Moda.e ,4 ,„ Rgara ,. 

^a^S^= 

the Assessor 27. the analogue data .s converted m to , *grta ' f °™ ™ ° °£ ^S>« Election item determines 
Bus goes through a suitable delay 2l^S2 tt «i«^ factors K1 

routines. As K1 and the data are both 8 bit n umbers ^^^^^^^1. that is necessary 
55 ^rs^^^^ 

TJs^ponent system of notification is choser ,,„ ^^^^^^^^ 
case, since the system is a normahsat.or \?^™£™™ n ™^^ {lM „ 00010000). which in 
60 sensing about the mean this must be defined as a m.dd le b. to the B brt numb e 1 < js 
turn means that between levels of K1 the mi ^ u r.^l^^„ ^lMB limits the maximum 
equivalent to only 4 bit resolution, and is eas.ly v.s.ble on ™^££££$£ a stari ng array, but if a 
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neighbour is slightly lower in response, requiring a higher normalisation correction. The next highest 
number possible is 2/16 which may well be 50% too large. andonlv 
The exponent type of multiply allows full 8 bit resolution to be ma.nta.ned at x ^16 and x 1/16, _and only 
suffers 7on ^quantLtion errors on the same scale at mu.tip.es of x 256 or x 1/256 ,f K1 ,s stored as 8 bus 

downstream at the start of every f >eld downstream in less than 1 field, and 

With small Arrays - thus 32 x 32 ^ r » J^f^J^™ ^ a Microprocessor (40) to access it and make 

25 data download. 

30 lp,aler»b,y .s cold as .he -»J M»h, <° ..mpertture of 

values computed and stored in Rrf «™"" "' S Q J K1 and K2 are computed and stored. At this stage 

the RAM areas in the Assessor are filled and all four sets of M anc ' Selector and the system is 

40 the Microprocessor then removes the restriction .mposed on the RAM B Jo* Selector, ^oine sy 

45 elements. This is a number which, .f the response is lower, - will then be called dead 



reference scenes). 
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CLAIMS 



,. A me,hodo f de,erm,n,n g ,he,naiv^ 
in a normalisation process, to the output values of °' * „ vie X ed scene, in which method:- 5 

adjacent regions; and rtrrortinn values for each element are calculated upon the basis that the 

range is divided into from two to ^6 regions. s gre much the same size 1 

T/5S?tS-* method ,s claimed in any o, .he preceding Calms and substantial es 
described hereinafter. p| PC tromaanetic radiation detector element array to improve * 

20 7. A method of processing the outpu of an elect ™wgnrtN ^ ^ pn£jr to 

the quality of the image b app'fng thereto the gain and offset correct, on factors 

its use to generate the image, is first normai.seo oy ^pp y h va|ue fal|s . 

appropriate both to that e.ement and to tM region of the range ^ matter< obtained an d stared the 
8 A method as claimed in Claim 7, in ^ ,ch *%"" 8 ^^e«emen7in the array, these then being in 

25 ^So=^ 

'Tl^claimedin^ 

the position in the range of that ou^^^^^i^S^w to the output value, and the corrected - 
30 applicable to that region for that element ' ^ t h ?se «e th e pp us> 

normalised - output value is passed ^^X^^Zt Si each element's output value to find in 
10. A method as claimed in Claim 9 in wn ' c ff h ;^f b S v e u S ^ g a seq uence of a corresponding plurality less 

which one of the plurality of regions ,t ^V^SiS!3t he output values defining the points where 

one of comparators in which are P^"^^ compared in all of these, and their outputs are 
35 adjacent regions meet, and the actual °^f^°^ on . the detector element's output value falls. 

-rr n r P u^ 

40 improve the quality of the image generate [f™*™™'™™ ^ rate the des ired image, this normalising 
output of each detector element pnor to _ that utput , use tc g f gctor and an offset correction 

means being means for applying to "■J^^^^fito applying means itself includes (Da 
factor each relevant to that part.cular ff ect ° r ' ^^et correction factors, for each detector element, for a 

50 correction factor and the appropriate offset correction ^ correctjon factors) . 

element whose output is to be two video stores (for holding two sequent*! 

14- Apparatus as ^f"^ the input to those stores, and the output 



rretmtoanimagegenerato^ 
55 15. Apparatus as cla.med in Claim 14 whe em men » 8 fof jts neighbouring elements, 

faulty element an average output value based the outjrt^ ^ hereinbefore 

apparatus as claimed in any of Claims 7 to 1 6. 
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